Medical and nursing clinician perspectives on the usability of the hospital electronic medical record: A qualitative analysis.

BACKGROUND: Electronic medical records (EMRs) have been widely implemented in Australian hospitals. Their usability and design to support clinicians to effectively deliver and document care is essential, as is their impact on clinical workflow, safety and quality, communication, and collaboration across health systems. Perceptions of, and data about, usability of EMRs implemented in Australian hospitals are key to successful adoption. OBJECTIVE: To explore perspectives of medical and nursing clinicians on EMR usability utilising free-text data collected in a survey. METHOD: Qualitative analysis of one free-text optional question included in a web-based survey. Respondents included medical and nursing/midwifery professionals in Australian hospitals (85 doctors and 27 nurses), who commented on the usability of the main EMR used. RESULTS: Themes identified related to the status of EMR implementation, system design, human factors, safety and risk, system response time, and stability, alerts, and supporting the collaboration between healthcare sectors. Positive factors included ability to view information from any location; ease of medication documentation; and capacity to access diagnostic test results. Usability concerns included lack of intuitiveness; complexity; difficulties communicating with primary and other care sectors; and time taken to perform clinical tasks. CONCLUSION: If the benefits of EMRs are to be realised, there are good reasons to address the usability challenges identified by clinicians. Easy solutions that could improve the usability experience of hospital-based clinicians include resolving sign-on issues, use of templates, and more intelligent alerts and warnings to avoid errors. IMPLICATIONS: These essential improvements to the usability of the EMR, which are the foundation of the digital health system, will enable hospital clinicians to deliver safer and more effective health care.

A digital health ecosystem ontology from the perspective of Australian consumers: a mixed-method literature analysis.

This study presents an ontology that scopes the digital health ecosystem from a consumer-centered perspective. We used a mixed-method analysis on a set of papers collected for a comprehensive review to identify common themes, components, and patterns that repeatedly emerge within Australian-based digital health studies. Three major and four child themes were identified as the foundational aspects of the proposed ontology. The child themes have more precise concept definitions, inherited and distinguishing attributes. Out of 179 recognized concepts, 33 were related to the Healthcare theme; 23 concepts formed a cluster of employed devices under the Technology theme; 40 concepts were associated with Use and Usability factors. 60 other concepts formed the cluster of the consumer-user theme. The theme of Digital Health was seen as being connected to 2 independent clusters. The main cluster embodied 21 extracted concepts, semantically related to "data, information, and knowledge," whilst the second cluster embodied concepts related to "healthcare." Different stakeholders can utilize this ontology to define their landscape of digitally enabled healthcare. The novelty of this work resides in capturing a consumer-centered perspective and the method we used in deriving the ontology - formalizing the results of a systematic review based on data-driven analysis methods.

BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning.

In many complex, real-world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high-quality e-courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web-based software platform and Delphi-style social processes. The result is an end-to-end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN-building and collaboration combine beneficially and cumulatively.

Understanding digital health ecosystem from Australian citizens' perspective: A scoping review.

BACKGROUND: Digital health (DH) and the benefits of related services are fairly well understood. However, it still is critical to map the digital health care landscape including the key elements that define it as an ecosystem. Particularly, knowing the perspectives of citizens on this digital transformation is an important angle to capture. In this review we aim to analyze the relevant studies to identify how DH is understood and experienced by Australian citizens and what they may require from DH platforms. MATERIALS AND METHODS: A scoping literature review was conducted across several electronic databases (ACM Digital Library, OVID, PubMed, Scopus, IEEE, Science Direct, SAGE), as well as grey literature. Additionally, citation mining was conducted to identify further relevant studies. Identified studies were subjected to eligibility criteria and the final set of articles was independently reviewed, analyzed, discussed and interpreted by three reviewers. RESULTS: Of 3811 articles, 98 articles met the inclusion criteria with research-based articles-as opposed to review articles or white papers- comprising the largest proportion (72%) of the selected literature. The qualitative analysis of the literature revealed five key elements that capture the essence of the digital health ecosystem interventions from the viewpoint of the Australian citizens. The identified elements were "consumer/user", "health care", "technology", "use and usability", "data and information". These elements were further found to be associated with 127 subcategories. CONCLUSIONS: This study is the first of its kind to analyze and synthesize the relevant literature on DH ecosystems from the citizens' perspective. Through the lens of two research questions, this study defines the key components that were found crucial to understanding citizens' experiences with DH. This understanding lays a strong foundation for designing and fostering DH ecosystem. The results provide a solid ground for empirical testing.

A National Survey of EMR Usability: Comparisons between medical and nursing professions in the hospital and primary care sectors in Australia and Finland.

BACKGROUND: Electronic Medical Record Systems (EMRs) are now part of nursing and medical professionals daily work in the acute and primary care sectors in Australia. Usability is an important factor in their successful adoption and impacts upon clinical workflow, safety and quality, communication, and collaboration. This study replicates a significant body of work conducted by Finnish researchers applying a usability focused survey to understand medical and nursing professionals' experiences in the Australian context. As we implement EMRs across health systems, their usability and design to support clinicians to effectively deliver and document care, is essential. METHODS: We conducted an observational study using a cross sectional survey, the National Usability-Focused HIS Scale (NuHISS) developed and validated by Finnish researchers. For this study 13 usability statements collected clinician impressions of EMRs related to technical quality, ease of use, benefits, and collaboration. We report the responses from medical and nursing professionals working in clinical practice settings in Australia, including primary care and hospital sectors in 2020. RESULTS: Nursing and medical professionals have different experiences with EMR usability. This depends on the sector they work in and the usability feature measured. In our sample, technical quality features were more positively experienced by doctors in the primary care sector than nurses as well as ease of obtaining patient information and prevention of errors. In the hospital sector nurses experiences with EMRs were more positive with respect to support for routine task completion, learnability, ease of obtaining patient information and entry of patient data. CONCLUSIONS: The NuHISS is a suitable tool for measuring the usability experiences of Australian clinicians and the EMRs utilised. Differences in usability experiences were noted between professional groups and sectors. A focus on the usability perspectives of clinicians when enhancing or developing EMR solutions is advocated.

Individuals vs. BARD: Experimental Evaluation of an Online System for Structured, Collaborative Bayesian Reasoning.

US intelligence analysts must weigh up relevant evidence to assess the probability of their conclusions, and express this reasoning clearly in written reports for decision-makers. Typically, they work alone with no special analytic tools, and sometimes succumb to common probabilistic and causal reasoning errors. So, the US government funded a major research program (CREATE) for four large academic teams to develop new structured, collaborative, software-based methods that might achieve better results. Our team's method (BARD) is the first to combine two key techniques: constructing causal Bayesian network models (BNs) to represent analyst knowledge, and small-group collaboration via the Delphi technique. BARD also incorporates compressed, high-quality online training allowing novices to use it, and checklist-inspired report templates with a rudimentary AI tool for generating text explanations from analysts' BNs. In two prior experiments, our team showed BARD's BN-building assists probabilistic reasoning when used by individuals, with a large effect (Glass' Δ 0.8) (Cruz et al., 2020), and even minimal Delphi-style interactions improve the BN structures individuals produce, with medium to very large effects (Glass' Δ 0.5-1.3) (Bolger et al., 2020). This experiment is the critical test of BARD as an integrated system and possible alternative to business-as-usual for intelligence analysis. Participants were asked to solve three probabilistic reasoning problems spread over 5 weeks, developed by our team to test both quantitative accuracy and susceptibility to tempting qualitative fallacies. Our 256 participants were randomly assigned to form 25 teams of 6-9 using BARD and 58 individuals using Google Suite and (if desired) the best pen-and-paper techniques. For each problem, BARD outperformed this control with very large to huge effects (Glass' Δ 1.4-2.2), greatly exceeding CREATE's initial target. We conclude that, for suitable problems, BARD already offers significant advantages over both business-as-usual and existing BN software. Our effect sizes also suggest BARD's BN-building and collaboration combined beneficially and cumulatively, although implementation differences decreased performances compared to Cruz et al. (2020), so interaction may have contributed. BARD has enormous potential for further development and testing of specific components and on more complex problems, and many potential applications beyond intelligence analysis.